1. Field of the Invention
The present invention relates to a technique for generating an image, a dynamic range of which is extended.
2. Description of the Related Art
In recent years, a so-called HDR (High Dynamic Range) technique for generating an image, a dynamic range of which is extended compared to an image obtained by normal imaging in an imaging apparatus such as a digital camera, is known. More specifically, an image processing method using the HDR technique can create an image (HDR image), a dynamic range of which is extended by combining a plurality of images captured to have different exposures.
However, in the HDR image created in this way, since a plurality of images are combined, multiple edges may appear at an edge portion of the obtained image due to the influences of image blurs and shakes acting on the imaging apparatus. Such multiple edge phenomenon in the created HDR image is notably confirmed especially at a boundary portion between high- and low-lightness portions.
The multiple edge phenomenon in the HDR image will be described in detail below using the drawings. Note that an example of creating an HDR image by combining two images having different exposures will be explained for the sake of simplicity.
For example, a framing in which a person as a main subject stands in a backlight relationship with respect to a background image with a high luminance value, as shown in FIG. 4A, will be examined below. In an image captured at that time, since a region where the main subject is rendered is a low-luminance region, when the main subject has moved or when the imaging apparatus has been shaken, multiple edges appear at a boundary portion between the main subject and background in an HDR image.
At this time, when a horizontal line 401 of the image shown in FIG. 4A is selected as a line of interest, in a low-exposure captured image and a high-exposure captured image, which are to be combined, pixel values DL and DH at x-coordinates of that horizontal line show behaviors shown in FIG. 5A. In FIG. 5A, the pixel values of the two images have different coordinates where the pixel values lower, due to the motion of the main subject or shakes of the imaging apparatus. Note that the exposure of the high-exposure captured image is four times that of the low-exposure captured image, and the pixel value of the high-exposure captured image is saturated in a high-luminance background region in the example shown in FIGS. 5A to 5F.
In order to combine two images having different exposures, the two images undergo gamma conversion or transformation. Since these images have different exposure conditions, the gamma conversion is done using different conversion characteristic curves for the respective images, as shown in FIG. 5B. In this manner, gamma conversion outputs GL and GH, which are converted in this way, show behaviors shown in FIG. 5C.
As can be seen from the gamma conversion outputs of the two images having different exposures, the high-exposure captured image whose background region is saturated exhibits gamma conversion output that is different from that of the low-exposure captured image in the background region. Also, in the low-exposure captured image, shadow saturation tends to occur in a low-luminance region generally, and a subject is not sharply expressed in that region. That is, an HDR image to be created preferably uses a low-exposure captured image for a high-luminance region, and a high-exposure captured image for a low-luminance region.
For this purpose, as an image having a reference exposure required to create an HDR image, a low-exposure captured image free from any saturated region is selected, and an adding coefficient K shown in FIG. 5D is decided according to a gamma conversion output GL of that image. The adding coefficient K is that which is decided in advance to decide a combining method of a low-exposure captured image and high-exposure captured image in an HDR image to be created, and a gamma output GO of the HDR image can be expressed by:GO=K×GL+KK×GH  (1)The adding coefficient K changes from 0 to 1 within a predetermined gamma conversion output range, as shown in, for example, FIG. 5D. Also, an adding coefficient KK is given by (1−K). More specifically, the two images, that is, the low-exposure captured image and high-exposure captured image are combined within a range of 0<K<1, only the low-exposure captured image is used in creation within a range of K=1, and only the high-exposure captured image is used in creation within a range of K=0.
The gamma conversion output of the HDR image created in this way shows a behavior shown in FIG. 5E. As can be seen from FIG. 5E, when the images having the different exposures, which are used to create the HDR image, suffer any image blurs or shakes of the imaging apparatus at their capturing timings, an image of a high-luminance region of the high-exposure captured image is combined in a low-luminance region. In this case, the multiple edge phenomenon called a pseudo edge occurs at a boundary between low- and high-luminance regions in the created HDR image.
As described above, when images to be combined suffer any image blurs or shakes of the imaging apparatus at their capturing timings, multiple edges often appear at a boundary between low- and high-luminance regions, resulting in an image quality drop of the HDR image and an unnatural feeling of the user.
Japanese Patent Laid-Open No. 2009-010566 discloses a technique for eliminating multiple edges in a combined image such that density values are averaged in the vicinity of a boundary of different intensities by setting one of a plurality of images having different exposures as an image having a reference exposure, and applying an averaging filter to the selected image.
Also, Japanese Patent Laid-Open No. 2005-223769 discloses a technique for detecting a difference between gamma conversion outputs of two images to be combined having different exposures in a given region, and creating an image by selecting a low-exposure captured image when the difference is larger than a threshold or a high-exposure captured image when the difference is smaller than another threshold.
However, with the technique of Japanese Patent Laid-Open No. 2009-010566 above, since the averaging processing is applied to the vicinity of a boundary of different intensities, the following unnatural combining processing may be unwantedly done while combining processing of images in the vicinity of the boundary. That is, since the averaging processing is applied to the vicinity of the boundary, a highlight-saturated high-exposure captured image is combined in a high-luminance region, or a shadow-saturated low-exposure captured image is combined in a low-luminance region. For this reason, since a white region is combined in the high-luminance region and a black region is combined in the low-luminance region to sandwich the boundary, an unnatural HDR image is likely to be created.
With the technique of Japanese Patent Laid-Open No. 2005-223769, fine pseudo edges are generated in a region with a small difference value of the gamma conversion outputs in a region which exhibits a steep luminance gradient.